Demultiplexing device

ABSTRACT

The demultiplexing apparatus  200  comprises a data obtainment unit  210  operable to obtain the MP4 file data  100 , a decoding unit  213  operable to demultiplex the MP4 file data  100  obtained by the data obtainment unit  210  into pairs of a moov  110  and a mdat  110  and pairs of a moof  130  and a mdat  140 , and decode those pairs pair by pair, a content duration specification unit  211   a  operable to specify the content total duration based on the content total duration information contained in the moov  110  of the MP4 file data  100  obtained by the data obtainment unit  210  and a playback unit  214  operable to display content total duration specified by the content duration specification unit  211   a.

TECHNICAL FIELD

The present invention relates to a demultiplexing apparatus fordemultiplexing multiplexed data where multi media data such as videodata and audio data are multiplexed, and a multiplexing apparatus formultiplexing multi media data.

BACKGROUND ART

An MP4 has been focused on as a file format for multiplexing and storingmulti media data (that is a content of video, audio, text, still pictureor the like) in these days.

The applicable range of the MP4 is expanding, for example, it isemployed as a file format of MPEG-4 that is a video compression codingmethod or JPEG 2000 that is a next generation version of JPEG that is anatural picture compression coding method.

This MP4 file format was initially composed of a data box wheremultiplexed content is stored and a header box where information on thecontent of the data part is stored.

Therefore, a multiplexing apparatus for generating MP4 file data frommulti media data obtains multi media data, performs coding andmultiplexing, and makes a header box to complete the MP4 file data afterfinishing processing on all the multi media data.

This multiplexing apparatus needs a lot of time to make the header boxafter finishing obtaining and processing multi media data when theinformation amount of the multi media data is a lot.

For example, in the case of a multiplexing apparatus made as a cellularphone with a video function, a time lag occurs at the time of receivinga call while videotaping because of making a header box whereinformation on all the videotaped contents are contained. Also, therewas a problem that the making process of a header box is cancelled inthe midstream and the MP4 file data cannot be completed when giving apriority to receiving a call.

Therefore, the MP4 file format is improved in a way that plural pairs ofa data box and a header box are connected to each other.

FIG. 1 is a data block diagram showing a basic file format of theconventional MP4 file data that is improved as mentioned above.

This MP4 file data 900 is composed of a non-fragment data part 900 a anda fragmented data part 900 b.

Non-fragment data part 900 a comprises a movie box 910 as theabove-mentioned header box (called “moov” from here) and a media databox 920 as the above-mentioned data box (called “mdat” from here).

The moov 910 is further composed of a movie header box 911 (called“mvhd” from here), a plurality of track box 912 (called “trak” fromhere), and a movie extend box 913 (called “mvex” from here).

The fragmented data part 900 b is made so that plural pairs of a moviefragment box 930 as the above-mentioned header box (called “moof” fromhere) and a media data box 940 as the above-mentioned data box (called“mdat” from here) are connected to each other.

The moof 930 is further composed of a movie fragment header box 931(called “mfhd” from here) and a plurality of track fragment box 932(called “traf” from here).

FIG. 2 is a data block diagram showing the detailed structure of themoov 910.

The mvhd 911 contains a movie duration field 911 a for storingnon-fragment content duration information indicating the duration neededfor playback of the content contained in the non-fragment data part 900a (the non-fragment content duration).

The trak 912 further contains a track header box 914 (called “tkhd” fromhere), and the tkhd 914 contains a track ID field 916 for storing trackidentification information for identifying a track and a track durationfield 917. Also, this track duration field 917 stores non-fragment trackduration information for indicating the duration needed for playback ofa track contained in the non-fragment data part 900 a (the non-fragmenttrack duration) indicated by the track identification information.

A track used here is a kind of contents and means video, audio, text orthe like.

The mvex 913 contains track extend box 915 (called “trex” from here) inproportion to the number of tracks contained in the fragmented data part900 b.

This trex 915 contains a track ID field 918 for storing trackidentification information for identifying a track and the first defaultduration field 919 for storing a default sample duration informationindicating the duration preset as a default to the track sample of thefragmented data part 900 b specified by the track identificationinformation.

A sample used here is a minimum unit of an MP4 file format. A samplemeans a frame or a picture when the track is video, while a sample meansaudio information, for example, of 20 msec when the track is audio.

FIG. 3 is a data block diagram showing the detailed structure of a moof930.

A traf 932 contained in the moof 930 further contains a track fragmentheader box 933 (called “tfhd” from here) and a plurality of trackfragment run box 936 (called “trun” from here).

This tfhd 933 contains a track ID field 934 for storing trackidentification information for identifying a track and the seconddefault duration field 935 for storing default sample durationinformation indicating the preset duration as a default to the sampleassociated with traf 932 stored in tfhd 933 out of the track sample ofthe fragmented data part 900 b specified by the track identificationinformation. The second default duration field 935 can be omitted here,and the first default duration field 919 is referred to in this case.

Also, trun 936 contains a sample duration field 937 for storing sampleduration information indicating the durations of respective samples(sample durations) contained in the tracks of the fragmented data part900 b specified by the above-mentioned track identification information.

In this way, the conventional multiplexing apparatus completes an MP4file data 900 every time it obtains and processes multi media data andmakes use of the effect in a real time recording by making MP4 fileformat contain a pair of a mdat 920 and a moov 910, and plural pairs ofa mdat 940 and a moof 930, in other words, by making a plural pairs of abox storing a multiplexed content and a box storing informationconcerning the content included in the earlier mentioned box areconnected to each other. And, the conventional multiplexing apparatusmade as a cellular phone with a videotaping function can avoid theoccurrence of a time lag at the time of receiving a call whilevideotaping.

FIG. 4 is a block diagram showing the structure of the conventionalmultiplexing apparatus for generating the above-mentioned MP4 file data900.

This multiplexing apparatus 700 comprises a video data analysis unit701, an audio data analysis unit 702, the first selector switch 703, thesecond selector switch 704, a control unit 708, a moof generation unit705, a moov generation unit 706, and a file generation unit 707.

The video data analysis unit 701 generates video specificationinformation indicating the time stamp or the size of the video data byobtaining and analyzing the video data. After that, the video dataanalysis unit 701 codes the video data, makes video coded data, andoutputs the video coded data and the video specification information.

The audio data analysis unit 702 generates audio specificationinformation indicating the time stamp or the size of the audio data byobtaining and analyzing the audio data. After that, the audio dataanalysis unit 702 codes the audio data, makes the audio coded data, andoutputs the audio coded data and the audio specification information.

The moov generation unit 706 generates and outputs the data (moov data)stored in a moov 910 according to the control from the control unit 708.

The moof output unit 705 generates and outputs the data (moof data)stored in a moof 930 according to the control from the control unit 708.

The control unit 708 obtains the video specification information fromthe video data analysis unit 701 and the audio specification informationfrom the audio data analysis unit 702, and outputs the data to be storedin the non-fragment data part 900 a that contains the moov data from thesecond selector switch 704 first and the data to be stored in thefragmented data part 900 b that contains the moof data next by switchingthe first selector switch 703 and the second selector switch 704 basedon the video specification information and audio specificationinformation.

The file generation unit 707 performs multiplexing processing on therespective data outputted from the second selector switch 704, andgenerates and outputs an MP4 file data 900.

FIG. 5 is a flow chart showing the operation concerning the generationof the moov data and the moof data of the conventional multiplexingapparatus 700.

First, the multiplexing apparatus 700 generates moov data (step S100)Next, the multiplexing apparatus 700 generates moof data (step S102).

After that, the multiplexing apparatus 700 judges whether or not thereremains unprocessed data in the inputted video data and audio data to becoded and multiplexed (step S104). When the multiplexing apparatus 700judges that there remains unprocessed data (step S104: Yes), itrepeatedly exerts the operation from the step S102 while it finishes theprocessing when it judges that there remains no unprocessed data (stepS104: No).

In this way, the conventional multiplexing apparatus 700 using the MP4file data 900 is effective for a real time recording because it makesmoof data in order.

By the way, the above-mentioned MP4 file data 900 is suitable for astreaming because it is made in a way that plural pairs of a data boxand a header box are connected to each other as shown in FIG. 1˜FIG. 3.In other words, the demultiplexing apparatus for playing back a contentcan play it back (perform download playback) in order before finishingdownloading all the MP4 file data 900 distributed as a stream byobtaining the MP4 file data 900 and performing demultiplexing.

FIG. 6 is a block diagram showing the structure of the conventionaldemultiplexing apparatus for playing back a content based on theabove-mentioned MP4 file data 900.

This demultiplexing apparatus 800 obtains and plays back the MP4 filedata 900, and comprises a data obtainment unit 810 operable to obtainthe MP4 file data 900, a decoding unit 813 operable to demultiplex anddecode the MP4 file data 900 obtained in the data obtainment unit 810, atime processing unit 811 operable to perform time processing based onthe obtained MP4 file data 900, a random access processing unit 812operable to perform a random access processing on the obtained MP4 filedata 900, a playback unit 814 operable to output video and audio basedon the data outputted from the decoding unit 813.

Also, the time processing unit 811 has a duration specification unit 811a operable to specify the duration needed for playback of part of thecontent contained in the MP4 file data 900 and a playback timespecification unit 811 b operable to specify playback time. Playbacktime used here means the time needed for playback from the starting timeto the present time when performing playback from the head of thecontent.

FIG. 7 is a block diagram showing the internal structure of the durationspecification unit 811 a.

The duration specification unit 811 a has the first demultiplexing unit821, the second demultiplexing unit 822, the third demultiplexing unit823, the first analysis unit 824, and the second analysis unit 825.

The first demultiplexing unit 821 demultiplexes the moov data from theMP4 file data 900 and outputs it on obtaining the MP4 file data 900.

The second demultiplexing unit 822 demultiplexes the data stored in mvhd911 (mvhd data) and the data stored in trak 912 (trak data) from themoov data and output them on obtaining the moov data.

The third demultiplexing unit 823 demultiplexes the data stored in tkhd914 (tkhd data) from the trak data and outputs it on obtaining the trakdata.

The first analysis unit 824 analyses the mvhd data and outputs thenon-fragment content duration information stored in the movie durationfield 911 a on obtaining mvhd data from the second demultiplexing unit822.

The second analysis unit 825 analyses the tkhd data and outputs thetrack identification information stored in the track ID field 916 onobtaining the tkhd data from the third demultiplexing unit 823, and italso outputs the non-fragment track duration information stored in thetrack duration field 917 according to the track identificationinformation.

The duration specification unit 811 a like this outputs the non-fragmentcontent duration information to the playback unit 814, and it alsooutputs, to the playback unit 814, the track identification informationand the non-fragment track duration information corresponding to thetrack of the non-fragment data part 900 a indicated by the trackidentification information.

After that, the playback unit 814 may display the non-fragment contentdurations or the non-fragment track durations for respective tracks asthe need arises when obtaining the non-fragment content durationinformation, the track duration information and the track identificationinformation. Also, the playback unit 814 displays the playback timeperforming playback of the video or the audio based on the specificationresult by the playback time specification unit 811 b.

FIG. 8 is a block diagram showing the structure of the random accessprocessing unit 812.

The random access processing unit 812 searches a sample corresponding toa target time as a target sample from the saved MP4 file data 900 onobtaining the target duration information indicating the target timefrom outside, and comprises a data storage unit 830 a, the firstdemultiplexing unit 830, the second demultiplexing unit 831, the thirddemultiplexing unit 832, the fourth demultiplexing unit 835, the fifthdemultiplexing unit 834, the first analysis unit 836, the secondanalysis unit 837, the third analysis unit 838, a track control unit833, a determination unit 839 and a control unit 840. The target timeused here means playback starting time when a user tries to performplayback from the middle of the content. For example, when a user triesto perform playback of the ten-minute content in total duration withoutviewing 2 minutes from the start, the target time is 2 minutes. The datastorage unit 830 a stores the MP4 file data 900 stored in the dataobtainment unit 810.

The first demultiplexing unit 830 demultiplexes the moov data and themoof data from the MP4 file data 900 and output them on obtaining theMP4 file data 900 from the data storage unit 830 a.

The third demultiplexing unit 832 demultiplexes data (mvex data) storedin the mvex 913 from the moov data and outputs the mvex data onobtaining the moov data.

The track control unit 833 outputs the track identification informationbased on the moov data on obtaining the moov data from the firstdemultiplexing unit 830.

The second demultiplexing unit 831 demultiplexes the data (traf data)stored in traf 932 corresponding to the track specified by the trackidentification information from the moof data and outputs the traf dataon obtaining the track identification information from the track controlunit 833.

The fourth demultiplexing unit 835 demultiplexes the data (tfhd data)stored in the tfhd 933 and the data (trun data) stored in the trun 936from the traf data and outputs them on obtaining the traf data from thesecond demultiplexing unit 831.

The fifth demultiplexing unit 834 demultiplexes the data (trex data)stored in the trex 915 corresponding to the track specified by the trackidentification information from the mvex data and outputs the trex dataon obtaining the mvex data from the third demultiplexing unit 832 andtrack identification information from the track control unit 833.

The first analysis unit 836 analyses the trex data and outputs thedefault sample duration information contained in the trex data as thefirst duration information on obtaining the trex data from the fifthdemultiplexing unit 834.

The second analysis unit 837 analyzes the tfhd data and outputs thesample duration information contained in the tfhd data as the secondduration information on obtaining the tfhd data from the fourthdemultiplexing unit 835.

The third analysis unit 838 analyses the trun data and outputs thesample duration information contained in the trun data as the thirdduration information on obtaining the trun data from the fourthdemultiplexing unit 835.

The determination unit 839 selects a piece of information out of thefirst duration information obtained from the first analysis unit 836,the second duration information obtained from the second analysis unit837 and the third duration information obtained from the third analysisunit 838 and then outputs the determined information as the durationinformation. First, the determination unit 839 gives the first priorityto the third duration information, gives higher priority to the secondduration information when the third duration information cannot beobtained because of its omission, selects the first duration informationwhen the second duration information cannot be obtained because of itsomission, and outputs the selected information.

Also, the determination unit 839 outputs the sample identificationinformation for identifying the sample corresponding to the durationinformation when outputting the duration information.

The control unit 840 adds durations indicated in the durationinformation outputted from the determination unit 839 in order onobtaining the target duration information showing the target time. Afterthat, the control unit 840 performs the above-mentioned addition untilthe addition result reaches to the target time indicated in the targetduration information, and outputs the sample identification informationobtained from the determination unit 839 as the target sampleidentification information when reaching to the target time.

FIG. 9 is a block diagram showing the internal structure of the trackcontrol unit 833.

The track control unit 833 comprises the sixth demultiplexing unit 841operable to obtain the moov data and demultiplex trak data from the moovdata, the seventh demultiplexing unit 842 further operable to obtain thetrak data and demultiplex tkhd data from the trak data, and an analysisunit 843 operable to specify and output the track identificationinformation by analyzing the tkhd data.

FIG. 10 is an illustration explaining the outline operation of therandom access processing unit 812.

As shown in this FIG. 10, the conventional random access processing unit812 specifies the target sample corresponding to the target time byadding the sample durations in order.

And, the random access processing unit 812 outputs the MP4 file data 900and the target sample identification information to the decoding unit813, and has the decoding unit 813 and the playback unit 814 exert theplayback of the content from the target sample indicated by the targetsample identification information.

The conventional demultiplexing apparatus 800 like this, when obtainingthe MP4 file data 900 distributed as a stream, can perform downloadplayback based on this and perform random access to all the stored MP4file data 900 by having the random access processing unit 812.

However, the above-mentioned conventional demultiplexing apparatus 800has the problem that it cannot display the total duration of all thecontents including contents in the fragmented data part 900 b while itcan display the durations needed for playback of the contents includedin the non-fragment data part when performing download playback of theMP4 file data 900, and thus it is not user-friendly.

FIG. 11 is an illustration explaining the problem in the conventionaldemultiplexing apparatus 800.

The demultiplexing apparatus 800 made as a cellular phone downloads theMP4 file data 900 distributed as a stream via a base station 990, andplays back the downloaded MP4 file data 900 in order. At that time, thedemultiplexing apparatus 800 displays the playback time, but it cannotspecify the duration needed for the playback of all the contents untilit obtains all the MP4 file data 900. As a result, it cannot display thetotal duration of the content during its download or playback. Forexample, a user of the demultiplexing apparatus 800 may have the fearthat the user cannot guess the fee in the rate system where fees arecharged according to the time spent using the demultiplexing apparatus800, or a user may have the fear that the power is cut in the middle ofthe download or the playback when a remaining battery capacity is nearexhaustion. In this way, the conventional demultiplexing apparatus 800cannot let the user know the total duration of the contents, and thus itlacks userfriendliness.

Therefore, the present invention is invented considering theabove-mentioned problems, and its purpose is to provide thedemultiplexing apparatus and the multiplexing apparatus with an improveduserfriendliness at the time of download playback of the content keepingthe effectiveness of the download playback and the real time recording.

DISCLOSURE OF INVENTION

In order to achieve the above-mentioned purpose, the demultiplexingapparatus concerning the present invention obtains multiplexed data of adigital copyrighted work starting from a header of the multiplexed dataand demultiplexes the multiplexed data, wherein the multiplexed data ismade in a way that plural pairs of a content storage box for storing apart of the multiplexed content and a related information storage boxfor storing related information on the part of the multiplexed contentare connected to each other, and that total duration information forspecifying a total duration needed for playback of all the contents arecontained in the header of the multiplexed data, the demultiplexingapparatus comprises: a data obtainment unit operable to obtain themultiplexed data; a demultiplexing processing unit operable todemultiplex the multiplexed data obtained by the data obtainment unitinto respective pairs; a total duration specification unit operable tospecify the total duration based on the total duration informationincluded in the header of the multiplexed data obtained by the dataobtainment unit; and a display unit operable to display the totalduration specified by the total duration specification unit. Themultiplexed data is composed of the first and the second parts havingthe pairs respectively. The total duration information is composed ofthe first duration information indicating the duration needed to playback the contents contained in the first part and the second durationinformation indicating the duration needed to play back the contentscontained in the second part. The total duration specification unit mayhave the feature that it specifies the total duration by adding theduration indicated by the first duration information to the durationindicated by the second duration information.

This enables the processing unit to demultiplex each pair of multiplexeddata, which makes it possible to perform download playback of thecontents from the multiplexed data distributed as a stream, let the userknow the total duration even during the download playback because thetotal duration is specified in the total duration specification unit andthe total duration is displayed by the display unit based on the totalduration information contained in the header of the multiplexed data,and improve the userfriendliness at the time of download playback of thecontents.

Also, the multiplexed data consists of a first part, or a first part anda second part having the pairs respectively according to the contentscontained in the multiplexed data, the total duration informationindicates a duration needed for playback of a content contained in thefirst part when the multiplexed data consists of the first part, thetotal duration information consists of first duration informationindicating a duration needed for playback of the content contained inthe first part and either of duration information needed for playback ofthe contents contained in the second part or second duration informationindicating that the durations are unknown when the multiplexed dataconsists of the first part and the second part, the total durationspecification unit judges whether the multiplexed data obtained by thedata obtainment unit consists of the first part, or the first part andthe second part, the total duration specification unit specifies theduration indicated in the total duration information as the totalduration when judging that the multiplexed data consists of the firstpart, the total duration specification unit further judges whether ornot the durations needed for playback of the contents contained in thesecond part is unknown based on the second duration information whenjudging that the multiplexed data consists of the first part and thesecond part, the total duration specification unit specifies the totalduration by adding the duration indicated by the first durationinformation to the duration indicated by the second duration informationwhen judging that the duration is known, the total durationspecification unit judges that it is impossible to specify the totalduration when judging that the duration is unknown, and the display unitinforms a user that the total duration is unknown when the totalduration specification unit judges that it is impossible to specify thetotal duration.

By doing this, the duration needed to play back the contents containedin the first part is determined as the total duration when themultiplexed data do not have the second part, on the other hand, whenthe multiplexed data is composed of the first and the second parts, thetotal duration can be correctly determined because the durations neededto play back the contents contained in the first part and the secondpart are added and the addition result is specified as the totalduration, plus whether or not the duration needed to play back thecontents contained in the second part is unknown based on the secondduration information contained in the demultiplexed data and thus it cannotify the user of only reliable total duration because it notifies theuser of “unknown” when the duration is unknown.

Also, the demultiplexing apparatus further comprising a playback unitoperable to play back the contents demultiplexed by the demultiplexingprocessing unit and a playback time specification unit operable tospecify playback time of the contents being played back by the playbackunit, and wherein the display unit calculates a ratio of the playbacktime, which is specified by the playback time specification unit, withrespect to the total duration specified by the total durationspecification unit, and moves a predetermined graphics according to theratio and displays the graphics.

By doing this, as the above-mentioned diagram moves according to theratio of the playback time to the total duration, the user can easilygrasp the playback condition of the contents.

Note that the present invention can also be realized as a data structureof the multiplexed data handled by the demultiplexing apparatus, astorage medium for storing the multiplexed data, a multiplexingapparatus for generating the multiplexed data, a demultiplexing methodperformed by the demultiplexing apparatus or a multiplexing methodperformed by the multiplexing apparatus, and a program for having thecomputer exert these methods.

Further Information about Technical Background to this Application

-   -   filed, is incorporated herein by reference.    -   Japanese Patent application No. 2002-126028 filed Apr. 26, 2002.    -   U.S. Patent application No. 60/378,644 filed May 9, 2002.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the invention willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the invention. In the

DRAWINGS

FIG. 1 is a data-block diagram showing the basic file format of the MP4file data in the conventional example.

FIG. 2 is a data block diagram showing the detailed structure of themoov in the MP4 file data in the conventional example.

FIG. 3 is a data block diagram showing the detailed structure of themoof in the MP4 file data in the conventional example.

FIG. 4 is a block diagram showing the structure of the conventionalmultiplexing apparatus for generating the MP4 file data in theconventional example.

FIG. 5 is a flow chart showing the operation concerning the generationof the moov data and the moof data of the multiplexing apparatus in theconventional example.

FIG. 6 is a block diagram showing the structure of the conventionaldemultiplexing apparatus for performing playback based on the MP4 filedata in the conventional example.

FIG. 7 is a block diagram showing the internal structure of the durationspecification unit in the conventional example.

FIG. 8 is a block diagram showing the structure of the random accessprocessing unit in the conventional example.

FIG. 9 is a block diagram showing the internal structure of the trackcontrol unit in the conventional example.

FIG. 10 is an illustration explaining the outline operation of therandom access processing unit in the conventional example.

FIG. 11 is an illustration explaining a problem in the conventionaldemultiplexing apparatus in the conventional example.

FIG. 12 is a data block diagram showing the file format of the MP4 filedata handled by the demultiplexing apparatus and the multiplexingapparatus in the embodiment.

FIG. 13 is a block diagram showing the structure of the demultiplexingapparatus for performing playback based on the MP4 file data in theconventional example.

FIG. 14 is a block diagram showing the internal structure of the contentduration specification unit of the time processing unit in theconventional example.

FIG. 15 is a block diagram showing the internal structure of the trackduration specification unit of the time processing unit in theconventional example.

FIG. 16 is a flow chart showing the characteristic operation of thecontent duration specification unit of the demultiplexing apparatus inthe conventional example.

FIG. 17 is a video display diagram showing the video displayed by theplayback unit of the demultiplexing apparatus in the conventionalexample.

FIG. 18 is a video display diagram showing another video displayed bythe playback unit of the demultiplexing apparatus in the conventionalexample.

FIG. 19 is a block diagram showing the structure of the multiplexingapparatus in the conventional example.

FIG. 20 is a flow chart showing the operation of the multiplexingapparatus in the conventional example.

FIG. 21 is a display diagram showing the syntax of the trex in theconventional example.

FIG. 22 is a block diagram showing the internal structure of the contentduration specification unit operable to handle the MP4 file data thatcontains the mehd in the conventional example.

FIG. 23 is a flow chart showing the characteristic operation of thecontent duration specification unit in the conventional example.

FIG. 24 is a display diagram showing the syntax of the mehd in theconventional example.

FIG. 25 is a data block diagram showing the file format of the moof ofthe MP4 file data handled by the demultiplexing apparatus concerning thevariation in the conventional example.

FIG. 26 is a block diagram showing the internal structure of the randomaccess processing of the demultiplexing apparatus concerning thevariation in the conventional example.

FIG. 27 is a block diagram showing the internal structure of the moofduration processing unit concerning the variation in the conventionalexample.

FIG. 28 is a block diagram showing the internal structure of the trafduration processing unit concerning the variation in the conventionalexample.

FIG. 29 is an illustration explaining the outline operation of therandom access processing unit concerning the variation in theconventional example.

FIG. 30 is a flow chart showing the operation of the random accessprocessing unit concerning the variation in the conventional example.

FIG. 31 is a display diagram showing the syntax of the tfhd concerningthe variation in the conventional example.

BEST MODE FOR CARRYING OUT THE INVENTION

The demultiplexing apparatus and the multiplexing apparatus in theembodiment of the present invention will be explained below withreference to figures.

FIG. 12 is a data block diagram showing the file format of the MP4 filedata handled by the demultiplexing apparatus and the multiplexingapparatus in the embodiment.

This MP4 file data 100 is composed of the non-fragment data part 100 aand fragmented data part 100 b and has the feature that the informationindicating the total duration of the contents contained in the data 100is stored in the non-fragment data part 100 a. The user can know thetotal duration in the demultiplexing apparatus because of the existenceof this information, which improves its userfriendliness. Also,apparatuses difficult to add this information such as multiplexingapparatus for recording in real time may omit adding the information.

The non-fragment data part 100 a is composed of a media data box 120(called “mdat” from here) where the multiplexed contents are stored andthe movie box 110 (called “moov” from here) where the information on thecontents of the mdat 120 are stored.

The moov 110 further consists of a movie header box 111 (called “mvhd”from here), plural track box 112 (called “trak” from here) and a movieextend box 113 (called “mvex” from here).

The fragmented data part 100 b is made in a way that plural pairs of amedia data box 140 (called “mdat” from here) where the multiplexedcontents are stored and a movie fragment box 130 (called “moof” fromhere) where the information on the contents of the mdat 140 are combinedwith each other like a chain. The moof 130 used here further consists ofa movie fragment header box (called “mfhd” from here) and plural trackfragment box (called “traf” from here) like the conventional example. Inother words, fragmented data part 100 b like this has the same structureas the conventional fragmented data part 900 b.

The mvhd 111 in the moov 110 of the non-fragmented data part 100 acontains a movie duration field 11 a for storing the non-fragmentcontent duration information indicating the duration needed to play backthe contents contained in the non-fragment data part 100 a (non-fragmentcontent duration).

The trak 112 contains a track header box 114 (called “tkhd” from here).This tkhd 114 used here contains the track ID field 116 for storing thetrack identification information for identifying the track and the trackduration field 117 for storing the non-fragment track durationinformation indicating the duration needed to play back the track of thenon-fragment data part 100 a (non-fragment track duration) indicated bythe track identification information.

And, the mvex 113 contains the same number of track extend box 115(called “trex” from here) as the number of the tracks contained in thefragmented data part 100 b, and a movie extend header box 113 a (called“mhex” from here).

The mhex 113 a contains a movie duration field 113 b for storing thefragmented content duration information indicating the duration(fragmented content duration) needed to play back the contents containedin the fragmented data part 100 b. The fragmented content durationinformation used here may show “0”, in this case, the fragmented contentduration information indicates that the fragmented content duration isunknown.

The trex 115 consists of a track ID field 118 for storing trackidentification information for identifying the track of the fragmenteddata part 100 b, the first default duration field 119 for storing thedefault sample duration information indicating the duration preset as adefault to the track sample of the fragmented data part 100 b to bespecified by the track identification information, and the trackduration field 115 a for storing the fragmented track durationinformation indicating the duration (fragmented track duration) neededto play back the track of the fragmented data part 100 b specified bythe track identification information.

In other words, the MP4 file data 100 in the embodiment has the featurethat a mvex 113 contains mhex 113 a where the fragmented contentduration information is stored, further, the trex 115 of the mvex 113contains the track duration field 115 a where the fragmented trackduration information is stored unlike the conventional example.

There is a case where the MP4 file data 100 is made up to only anon-fragment data part 100 a when the information amount of the contentscontained there is small, in this case, the mvex 113 of the moov 110 isomitted from the structure.

FIG. 13 is a block diagram showing the structure of the demultiplexingapparatus 200 for playing back the contents based on the above-mentionedMP4 file data 100 in the embodiment.

This demultiplexing apparatus 200 obtains and plays back the MP4 filedata 100 and comprises a data obtainment unit 210 operable to obtain theMP4 file data 100 from the non-fragment data part 100 a in order, adecoding unit 213 operable to demultiplex the MP4 file data 100 obtainedin the data obtainment unit 210 into each pair of a moov 110 and a mdat120 or each pair of a moof 130 and a mdat 140 in order and decode them,a time processing unit 211 operable to perform time processing based onthe MP4 file data 100 obtained in the data obtainment unit 210, a randomaccess processing unit 212 operable to perform a random access to theobtained MP4 file data 100, and a playback unit 214 operable to outputvideo or audio based on the data outputted by the decoding unit 213.

The random access processing unit 212 stores all the MP4 file data 100obtained in the data obtainment unit 210, searches the samplecorresponding to the target time as the target sample out of the storedMP4 file data 100 when obtaining the target duration informationindicating the target time from outside, and has the decoding unit 213and the playback unit 214 play back the contents from the target sample.

Also, the time processing unit 211 consists of a content durationspecification unit 211 a operable to specify the total duration neededto play back all the contents contained there based on the MP4 file data100, a track duration specification unit 211 b operable to specify theduration needed to play back all the tracks contained in the MP4 filedata 100 track by track, and a playback time specification unit 211 coperable to specify the playback duration.

FIG. 14 is a block diagram showing the internal structure of the contentduration specification unit 211 a of the time processing unit 211.

The content duration specification unit 211 a consists of the firstdemultiplexing unit 221, the second demultiplexing unit 222, the thirddemultiplexing unit 224, the first analysis unit 223, the secondanalysis unit 225, and an addition unit 226.

The first demultiplexing unit 221 obtains the MP4 file data 100, anddemultiplexes and outputs the data (moov data) stored in the moov 110from the MP4 file data 100.

The second demultiplexing unit 222, on obtaining the moov data,demultiplexes the data (mvhd data) stored in the mvhd 111 and the data(mvex data) stored in the mvex 113 from the moov data and outputs them.

The third demultiplexing unit 224, on obtaining the mvex data,demultiplexes the data (mhex data) stored in the mhex 113 a from themvex data and outputs it.

The first analysis unit 223, on obtaining the mvhd data from the seconddemultiplexing unit 222, analyzes the mvhd data and outputs thenon-fragmented content duration information stored in the movie durationfield 11 a.

The second analysis unit 225, on obtaining the mhex data from the thirddemultiplexing unit 224, analyzes the mhex data and outputs thefragmented content duration information stored in the movie durationfield 113 b.

The addition unit 226, on obtaining the non-fragment content durationinformation and the fragmented content duration information from thefirst analysis unit 223 and the second analysis unit 225, adds thenon-fragment content duration indicated in the non-fragment contentduration information to the fragmented content duration indicated in thefragmented content duration information, and outputs the content totalduration information indicating the duration after addition (totalcontent duration). Also, the addition unit 226 outputs the non-fragmentcontent duration information as the total content duration informationwhen the fragmented content duration information shows “0”.

In this way, the content duration specification unit 211 a in theembodiment specifies the total content duration contained in thenon-fragment data part 100 a and the fragmented data part 100 b of theMP4 file data 100 and outputs the total content duration informationindicating the total content duration.

FIG. 15 is a block diagram showing the internal structure of the trackduration specification unit 211 b of the time processing unit 211.

The track duration specification unit 211 b consists of the firstdemultiplexing unit 231, the second demultiplexing unit 232, the thirddemultiplexing unit 233, the fourth demultiplexing unit 235, the firstanalysis unit 234, the second analysis unit 236 and the addition unit237.

The first demultiplexing unit 231 obtains the MP4 file data 100,demultiplexes the moov data from the MP4 file data 100 and outputs it.

The second demultiplexing unit 232, on obtaining the moov data,demultiplexes the data (trak data) stored in the trak 112 and the mvexdata from the moov data and outputs them.

The third demultiplexing unit 233, on obtaining the trak data,demultiplexes the data (tkhd data) stored in the tkhd 114 from the trakdata and outputs it.

The first analysis unit 234, on obtaining the tkhd data from the thirddemultiplexing unit 233, analyzes the tkhd data and outputs the trackidentification information stored in the track ID field 116 of the tkhd114. Further, the first analysis unit 234 extracts the non-fragmenttrack duration information corresponding to the track specified by thetrack identification information from the track duration field 117 ofthe tkhd 114 and outputs it.

The fourth demultiplexing unit 235, on obtaining the mvex data from thesecond demultiplexing unit 232 and track identification information fromthe first analysis unit 234, demultiplexes the data (trex data) storedin the trex 115 corresponding to the track specified in the trackidentification information from the mvex data and outputs it.

The second analysis unit 236, on obtaining the trex data from the fourthdemultiplexing unit 235, analyzes the trex data and outputs thefragmented track duration information stored in the track duration field115 a.

The addition unit 237, on obtaining the non-fragment track durationinformation from the first analysis unit 234 and the fragmented trackduration information from the second analysis unit 236, adds theduration indicated in the non-fragment track duration information to theduration indicated in the fragmented track duration information, andoutputs the total track duration information indicating the durationafter addition (total track duration).

In this way, the track duration specification unit 211 b in the presentembodiment specifies track total duration contained in the non-fragmentdata part 100 a and the fragmented data part 100 b of the MP4 file data100 track by track and outputs the track identification informationindicating these tracks and total track duration information indicatingthe total track duration of these tracks.

FIG. 16 is a flow chart showing the characteristic operation of thecontent duration specification unit 211 a of the demultiplexingapparatus 200 in this embodiment.

First, the content duration specification unit 211 a obtains the MP4file data 100 (step S200), analyses the mvhd 111 of the data 100 andspecifies the non-fragment content duration T1 (step S202).

Next, the content duration specification unit 211 a judges whether ornot the mvex 113 is contained in the moov 110 of the data 100 obtainedin the step S200 (step S204).

The content duration specification unit 211 a used here, on detectingthe mvex 113 (step S204: Yes), judges that the MP4 file data 100 iscomposed of the non-fragment data part 100 a and the fragmented datapart 100 b, further, by analyzing the mhex 113 a contained in the mvex113 (step S206), judges whether or not the fragmented content durationinformation stored in the movie duration field 113 b shows “0” (stepS208).

The content duration specification unit 211 a, on judging that thefragmented content duration information does not shows “0” (step S208:No), specifies the fragmented content duration T2 from the fragmentedcontent duration information (step S210).

After that, the content duration specification unit 211 a adds thenon-fragment content duration T1 to the fragmented content duration T2,calculates the total content duration T (step S212), and outputs thetotal content duration information indicating the total content durationT (step S214).

Meanwhile, when the mvex 113 is not detected in the step S204 (stepS204: No), the content duration specification unit 211 a judges that theMP4 file data 100 is made from the non-fragment data part 100 a, handlesthe non-fragment content duration T1 as the total content duration T(step S220), and outputs the total content duration information (stepS214).

After that, when the content duration specification unit 211 a judgesthat the fragmented content duration information shows “0” in the stepS208 (step S208: Yes), it judges that it is impossible to calculate thetotal content duration T (step S216), and outputs the total durationunknown information indicating that the total content duration T isunknown (step S218).

Here, the playback unit 214 of the demultiplexing apparatus 200 in thisembodiment outputs video or audio based on the data from the decodingunit 213, displays the playback time specified by the playback timespecification unit 211 c, and displays the total content durationindicated by the total content duration information when the totalcontent duration information is outputted from the content durationspecification unit 211 a, or indicates that the total content durationis unknown when the total duration unknown information is outputted fromthe content duration specification unit 211 a.

FIG. 17 is a video display diagram showing the video displayed by theplayback unit 214 when the total content duration information isoutputted.

The playback unit 214 displays the display screen P1 for displaying theplayback video, and displays a playback video display unit P2 where theplayback video is displayed, a slider bar P3, an operation button P4,and a time display unit P5 on the display screen P1.

The total content duration such as “00:02:30” is displayed in the rightpart of the time display unit P5, and the playback time such as“00:00:30” is displayed in the left part of the time display unit P5.

The slider bar P3 slides from the left to the right on the line Ln1according to the ratio of the playback time to the total contentduration.

FIG. 18 is a video display diagram showing the video displayed by theplayback unit 214 when the total duration unknown information isoutputted.

The playback unit 214 displays the display screen P1 contains theplayback video display unit P2, the slider bar P3, the operation buttonP4 and the time display unit P5 like mentioned above, but it indicatesletters or signs such as “--:--:--” indicating that the total contentduration is unknown in the right part of the time display unit P5. Also,the playback unit 214 displays the playback time such as “00:00:30” inthe left part of the time display unit P5.

Also, the slider bar P3 slides from the left to the right on the lineLn1 in the FIG. 18 according to the ratio of the playback time to thepredetermined time, and repeatedly exerts sliding from the left to theright in the FIG. 18 each time the above-mentioned predetermined timepasses.

Also, the playback unit 214, when obtaining the track identificationinformation and the total track duration information from the trackduration specification unit 211 b, displays the total track durationindicated by the total track duration information for each trackindicated in the track identification information as the need arises.

In this way, the demultiplexing apparatus 200 in the embodiment canperform download playback when starting the download of the MP4 filedata 100, display the total content duration based on the total contentduration information stored in the moov 110 of the MP4 file data 100even before completing download of all the MP4 file data 100, andimprove the userfriendliness.

Next, the multiplexing apparatus 300 for making the MP4 file data 100 inthe embodiment will be explained.

FIG. 19 is a block diagram showing the structure of the multiplexingapparatus 300 in the embodiment.

The multiplexing apparatus 300 in this embodiment comprises a video dataanalysis unit 301, an audio data analysis unit 302, the first selectorswitch 303, the second selector switch 304, a control unit 308, a moofgeneration unit 305, a moov generation unit 306 and a file generationunit 307.

The video data analysis unit 301 generates the video specificationinformation indicating the time stamp and the size of the video data byobtaining and analyzing the video data. After that, the video dataanalysis unit 301 codes the video data and makes the video coded data,and then outputs the video coded data and the video specificationinformation.

The audio data analysis unit 302 generates the audio specificationinformation indicating the time stamp and the size of the audio data byobtaining and analyzing the audio data. After that, the audio dataanalysis unit 302 codes the audio data and makes the audio coded data,and then outputs the audio coded data and the audio specificationinformation.

The moov generation unit 306 generates the moov data based on thecontrol from the control unit 308 and outputs it. The moov generationunit 306 in the embodiment here generates mhex 113 a in the mvex 113 ofthe moov data.

The moof generation unit 305 generates the moof data based on thecontrol from the control unit 308 and outputs it.

The control unit 308 obtains the video specification information fromthe video data analysis unit 301 and the audio specification informationfrom the audio data analysis unit 302, outputs the data to be stored inthe non-fragment data part 100 a that contains the moov data first fromthe second selector switch 304 by switching the first selector switch303 and the second selector switch 304 based on the video specificationinformation and the audio specification information, and outputs thedata to be stored in the fragmented data part 100 b that contains themoof data next.

The control unit 308 in the embodiment here, when having the moovgeneration unit 306 output the moov data, has the moov generation unit306 generate the moov data that enables the fragmented content durationinformation to be stored in the movie duration field 113 b of the mhex113 a to show “0”, and has the moov generation unit 306 output the moovdata.

After that, the control unit 308 adds, in order, the durations of therespective samples to be stored in the fragmented data part 100 b of thevideo coded data and the audio coded data based on the videospecification information and the audio specification information.

Next, the control unit 308 has the moof generation unit 305 output themoof data for all the inputted audio data and video data, regards theduration added as mentioned above as the fragmented content duration,and output the fragmented content duration information indicating thefragmented content duration to the moov generation unit 306.

After that, the moov output unit 306 that obtained the fragmentedcontent duration information from the control unit 308 outputs, to thefile generation unit 307, the fragmented content duration informationand the position specification information indicating the position ofthe movie duration field 113 b of the mhex 113 a where the informationis stored.

The file output unit 307, on obtaining the fragmented content durationinformation and the position specification information from the moovgeneration unit 306, overwrites the obtained fragmented content durationinformation on the movie duration field 113 b of the mhex 113 aindicated in the position specification information. In other words, thefile generation unit 307 changes the fragmented content durationinformation showing “0” to the later-obtained fragmented contentduration information, and then it completes the MP4 file data 100.

FIG. 20 is a flow chart showing the operation of the multiplexingapparatus 300 in the embodiment.

First, the multiplexing apparatus 300 generates the moov data thatcontains the fragmented content duration information showing “0” (stepS300). In other words, the multiplexing apparatus 300 generates aprovisional moov data first.

Next, the multiplexing apparatus 300 generates the moof data (stepS302).

After that, the multiplexing apparatus 300 judges whether or not thereremains unprocessed data in the inputted video data and audio data to becoded and multiplexed (step S304). The multiplexing apparatus 300repeatedly exerts the operation from step S302 when judging that thereremains unprocessed data (step S304: Yes), while it overwrites thefragmented content duration information (step S306) based on theaddition result of the sample durations on the moov data and thenfinishes the processing when judging that there remains no unprocesseddata (step S302: No).

As the multiplexing apparatus 300 in the embodiment like this generatesthe MP4 file data 100 made in a way that a pair of a moov 110 and a mdat120 and at least a pair of a moof 130 and a mdat 140 are connected toeach other according to the information amounts of the contents, themultiplexing apparatus 300 is effective for a real time recording andcan store the fragmented content duration information needed to improvethe userfriendliness of the demultiplexing apparatus 200 in the moov110.

FIG. 21 is a display diagram showing the syntax of the trex.

To the track duration field 115 a of the trex 115, 32 bit or 64 bit isassigned according to the version, and 32 bit is assigned to the firstdefault duration field 119. Also, a time scale is indicated by the mvhd111.

While the mhex 113 a that contains the fragmented content durationinformation is stored in the mvex 113 of the MP4 file data 100 in thisembodiment, a box (called “mehd” from here) that contains the totalcontent duration information may be stored instead of in the mhex 113 a.

FIG. 22 is a block diagram showing the internal structure of the contentduration specification unit operable to specify the total contentduration based on the MP4 file data that contains the mehd.

The content duration specification unit 211 a′ consists of the firstdemultiplexing unit 211′, the second demultiplexing unit 222′, the thirddemultiplexing unit 224′, and an analysis unit 225′. The firstdemultiplexing unit 221′ obtains the MP4 file data, demultiplexes themoov data from the MP4 file data, and outputs the moov data. The seconddemultiplexing unit 222′, on obtaining the moov data, demultiplexes themvex data from the moov data and outputs the mvex data. The thirddemultiplexing unit 224′, on obtaining the mvex data, demultiplexes themehd data stored in the above-mentioned mehd from the mvex data andoutputs the mehd data. The analysis unit 225′, on obtaining the mehddata from the third demultiplexing unit 224′, analyzes the mehd data andoutputs the total content duration information.

FIG. 23 is a flow chart showing the characteristic operation of theabove-mentioned content duration specification unit 211 a′.

First, the content duration specification unit 211 a′ obtains theabove-mentioned MP4 file data (step S260), and judges whether or not themehd is contained in the data (step S262).

Here, the content duration specification unit 211 a′, on detecting themehd (step S262: Yes), analyzes the mehd (step S264), and judges whetheror not the total content duration information stored in the mehd shows“0” (step S266).

After that, the content duration specification unit 211 a′, when judgingthat the total content duration information does not show “0” in thestep S266 (step S266: No), outputs the total content durationinformation (step S268). On the other hand, the content durationspecification unit 211 a′ judges that it is impossible to specify thetotal content duration (step S270) and outputs total duration unknowninformation indicating that the total content duration is unknown (stepS272) when judging that the mehd is not contained in the data in thestep S262 (step S262: No) or when judging that the total contentduration information shows “0” in the step S266 (step S266: Yes).

In this way, by storing the mehd in the MP4 file data instead of themhex 113 a, the content duration specification unit 211 a′ can easilyspecify the total content duration by analyzing the mehd data withoutadding the content duration of the non-fragment data part 100 a to thecontent duration of the fragmented data part 100 b so as to specify thetotal content duration.

On the other hand, the control unit 308 of the multiplexing apparatus300, when generating the moov data in the moov generation unit 306,generates the moov data so that the total content duration informationto be stored in the mehd shows “0” in the moov generation unit 306first, and adds respective sample durations to be stored in thenon-fragment data part 100 a and the fragmented data part 100 b of thevideo coded data and the audio coded data in order based on the videospecification information and the audio specification information. Afterthat, the control unit 308, on outputting the moof data for all theinputted audio data and video data from the moof generation unit 305,regards the duration added as mentioned above as the total contentduration, and outputs the total content duration information indicatingthe duration to the moov generation unit 306. In this way, the filegeneration unit 307 of the multiplexing apparatus 300 generates the MP4file data 100 for storing the above-mentioned mehd.

FIG. 24 is a display diagram showing the syntax of the mehd.

To the movie duration field for storing the total content durationinformation of the mehd, 32 bit or 64 bit is assigned according to theversion. For example, 32 bit is assigned when the version shows “0”, or64 bit is assigned when the version shows “1”. Also, the time scale isindicated by the mvhd.

Note that the mhex 113 a can have the syntax as shown in FIG. 24.

Also, the fragmented content duration is unknown when the fragmentedcontent duration information shows “0” in the above-mentionedembodiment, but the value can be the one except “0” such as a negativevalue.

(Variation)

Next, the variation of the demultiplexing apparatus 200 in theembodiment will be explained.

The above-mentioned demultiplexing apparatus 200 uses the MP file datawhose structure of the non-fragment data part differs from theconventional file format, but the demultiplexing apparatus concerningthis variation further uses the MP4 file data whose structures of thenon-fragment part and the fragmented part differ from the conventionalfile format.

FIG. 25 is a data block diagram showing the file format of the moof ofthe MP4 file data used by the demultiplexing apparatus concerning thisvariation.

The moof 130′ is composed of a mfhd 131 and plural traf 132.

The mfhd 131 is composed of an entry counter field 131 a and pluralduration entry field 131 b. Further, in the duration entry field 131 b,a track ID field 131 c for storing track identification information foridentifying a track and a moof duration field 131 d for storing the moofduration information indicating the track duration (moof duration)specified by the track identification information in the mdat 140pairing up with the moof 130′.

The traf 132 is composed of a track fragment header box 133 (called“tfhd” from here) and plural track fragment run box 136 (called “trun”from here), the tfhd 133 contains a traf duration field 135 where thetraf duration information is stored and the second default durationfield 134 where the default sample duration information is stored.

The traf duration information stored in the traf duration field 135indicates the addition result (traf duration) of the sample durationindicated in the sample duration information contained in the trun 136of the traf 132, and the default sample duration information stored inthe second default duration field 134 indicates the duration to the trafduration set as a default.

FIG. 26 is a block diagram showing the internal structure of the randomaccess processing unit of the demultiplexing apparatus concerning thisvariation.

The random access processing unit 400 can reduce the calculation amountcompared to the conventional random access processing unit 812 andquickly search the target sample. The random access processing unit 400comprises a data storage unit 401 operable to store the MP4 file data100′ with the moof 130′ shown in FIG. 25, a moof duration processingunit 420 operable to specify the moof duration, a traf durationprocessing unit 430 operable to specify the traf duration, a sampleduration processing unit 440 operable to specify the sample duration,and a main control unit 410 operable to control these processing units.

The main control unit 410, on obtaining the MP4 file data 100′ and thetarget duration information indicating the target time, analyses thestructure of the MP4 file data 100′, controls the moof durationprocessing unit 420, the traf duration processing unit 430 and thesample duration processing unit 440, and outputs the target sampleidentification information indicating the target sample according to theabove-mentioned target time by performing addition processing of themoof duration, the traf duration and the sample duration specified byrespective one of these processing units.

FIG. 27 is a block diagram showing the internal structure of the moofduration processing unit 420.

The moof duration processing unit 420 comprises the first demultiplexingunit 421, the second demultiplexing unit 423, a track control unit 422,and an analysis unit 424.

The first demultiplexing unit 421, on obtaining the MP4 file data 100′with the moof 130′ in FIG. 25, demultiplexes the moov data and the moofdata from the MP4 file data 100′ and outputs them.

The second demultiplexing unit 423, on obtaining the moof data from thefirst demultiplexing unit 421, demultiplexes the data (mfhd data) storedin the mfhd 131 from the moof data and outputs it.

The track control unit 422, on obtaining the moov data from the firstdemultiplexing unit 421, outputs the track identification informationbased on the moov data.

The analysis unit 424 obtains the mfhd data from the seconddemultiplexing unit 423 and the track identification information fromthe track control unit 422, analyzes the mfhd data, extracts the moofduration information of the track specified by the track identificationinformation from the mfhd data, and outputs the moof durationinformation.

FIG. 28 is a block diagram showing the internal structure of the trafduration processing unit 430.

The traf duration processing unit 430 consists of the firstdemultiplexing unit 431, the second demultiplexing unit 433, the thirddemultiplexing unit 434, a track control unit 432 and an analysis unit435.

The first demultiplexing unit 431, on obtaining the MP4 file data 100′,demultiplexes the moov data and the moof data from the MP4 file data100′ and outputs them.

The track control unit 432, on obtaining the moov data from the firstdemultiplexing unit 431, outputs the track identification informationbased on the moov data.

The second demultiplexing unit 433 obtains the moof data from the firstdemultiplexing unit 431 and the track identification information fromthe track control unit 432, analyzes the moof data, demultiplexes thetraf data corresponding to the track specified in the trackidentification information from the moof data, and outputs the trafdata.

The third demultiplexing unit 434, on obtaining the traf data from thesecond demultiplexing unit 433, demultiplexes the tfhd data from thetraf data and outputs the tfhd data.

An analysis unit 435, on obtaining the tfhd data from the thirddemultiplexing unit 434, analyzes the tfhd data, extracts the trafduration information from the tfhd data and outputs the traf durationinformation.

Also, the sample duration processing unit 440 demultiplexes the trundata track by track by performing the predetermined demultiplexing onthe MP4 file data 100′, extracts the sample duration information fromthe trun data and outputs the sample duration information.

FIG. 29 is an illustration explaining the outline operation of therandom access processing unit 400.

The (a) of FIG. 29 is a data block diagram indicating the stream of theMP4 file data 100′. The stream like this is formed for each sample, andeach sample contains a flag indicating that the sample is seekable(readable first).

The (b) of FIG. 29 is a data block diagram showing the segment structureof the MP4 file data 100′.

The file format of the MP4 file data 100′ contains the moov 110 and themoof 130′ as mentioned above, the moof 130′ contains a traf 132.

The total content duration contained in the MP4 file data 100′ is shownas the addition result of the non-fragment content duration A and thefragmented content duration B.

Further, the fragmented content duration B is shown as the additionresult of the moof duration C₁, C₂, and so on indicated by each moof130′ of the fragmented data part, and the moof durations C₁, C₂, and soon are shown as the addition result of the traf durations D₁, D₂, and soon indicated by each traf 132 of the respective moof 130′.

The (c) of FIG. 29 is an illustration explaining the seek operation ofthe random access processing unit 400.

The random access processing unit 400, when obtaining the targetduration information indicating the target time, adds the fragmentedcontent duration B to the non-fragment content duration A first. Whenthe addition result becomes equal to the target time, the random accessprocessing unit 400 specifies the lastly-obtained sample of the MP4 filedata 100′ as the target sample and outputs the target sampleidentification information indicating the target sample.

On the other hand, when the addition result exceeds the target time, therandom access processing unit 400 adds the moof durations C₁, C₂, and soon to the non-fragment content duration A in a time order. When theaddition result becomes equal to the target time, the random accessprocessing unit 400 specifies the last sample corresponding to thelastly-added moof duration as the target sample and outputs the targetsample identification information indicating the target sample.

Also, when the addition result exceeds the target time, the randomaccess processing unit 400 adds the traf durations D₁, D₂, and so on ina time order to the addition result immediately before exceeding thetarget time. When the addition result becomes equal to the target time,the random access processing unit 400 specifies the last samplecorresponding to the lastly-added traf duration as the target sample,and outputs the target sample identification information indicating thesample.

In addition, when the addition result exceeds the target time, therandom access processing unit 400 adds the sample durations E₁, E₂ andso on in a time order to the addition result immediately beforeexceeding the target time. Here, when the addition result becomes equalto the target time, the random access processing unit specifies thesample corresponding to the lastly-added sample duration as the targetsample, and outputs the target sample identification informationindicating the target sample.

FIG. 30 is a flow chart showing the operation of the random accessprocessing unit 400.

First, the random access processing unit 400 adds the fragmented contentduration B to the non-fragment content duration A, and judges whether ornot the addition result becomes equal to the target time X (step S400).

When the addition result is judged to be equal as a result (step S400:Yes), the random access processing unit 400 judges that the targetsample is the last sample of the MP4 file data (step S402).

On the other hand, when the addition result is judged to be unequal(step S400: No), the random access processing unit 400 regards thevariable N as 1 (step S404) and calculates the duration a using thenon-fragment content duration A and the moof duration C_(n) based on thefollowing formula (step S406).$a = {A + {\sum\limits_{n = 1}^{N}\quad C_{n}}}$

Here, the random access processing unit 400 judges whether or not theduration a is equal to the target time X (step S408), and further judgesthat the target sample is the last sample corresponding to the moofduration C_(n) (step S410) when judging that it is equal (step S408:Yes).

Also, the random access processing unit 400, when judging that theduration a is not equal to the target time X (step S408: No), furtherjudges whether or not the duration a is bigger than the target time X(step S412).

The random access processing unit 400, when judging that the duration ais not bigger than the target time X (step S412: No), regards N as N+1(step S414), further repeatedly exerts the operation from the step S406.The above-mentioned formula shows the addition result of thenon-fragment content duration A and the moof durations C₁˜C_(n), but therandom access processing unit 400 solves the above-mentioned formula bytemporally storing the addition result and further adding the C_(n)corresponding to the N to the saved result of the addition processingmentioned above each time N is incremented by 1 instead of exertingaddition processing of the above-mentioned formula each time N isincremented.

When the duration a is judged to be bigger than the target time X (stepS412: Yes), the random access processing unit 400 regards variable M as1 (step S416) first, and calculates the duration b using thenon-fragment content duration A, the moof duration C_(n), and the trafduration D_(m) based on the following formula (step S417).$b = {A + {\sum\limits_{n = 1}^{N - 1}\quad C_{n}} + {\sum\limits_{m = 1}^{M}\quad D_{m}}}$

Here, the random access processing unit 400 judges whether or not theduration b is equal to the target time X (step S418), when judging thatit is equal (step S418: Yes), the target sample is judged to be the lastsample corresponding to the traf duration D_(m) (step S420).

Also, the random access processing unit 400, when the duration b isjudged to be unequal to the target time X (step S418: No), furtherjudges whether or not the duration b is bigger than the target time X(step S422).

The random access processing unit 400, when judging the duration b issmaller than the target time X (step S422: No), regards M as M+1 (stepS424), further repeatedly exerts the operation from the step S417. And,when judging the duration b is bigger than the target time X (step S422:Yes), the random access processing unit 400 regards variable L as 1(step S426) first, and then calculate the duration c using thenon-fragment content duration A, the moof duration C_(n), the trafduration D_(m) and the sample time E_(l) based on the following formula(step S428).$c = {A + {\sum\limits_{n = 1}^{N - 1}\quad C_{n}} + {\sum\limits_{m = 1}^{M - 1}D_{m}} + {\sum\limits_{l = 1}^{L}\quad E_{l}}}$

Here, the random access processing unit 400 judges whether or not theduration c is equal to the target time X (step S430), and judges thatthe target sample is the sample corresponding to the sample durationE_(l) (step S432) when judging that duration c is equal to the targettime X (step S430: Yes).

Also, the random access processing unit 400, when judging that theduration c is unequal to the target time X (step S430: No), furtherjudges whether or not the duration c is bigger than the target time X(step S434).

The random access processing unit 400, when judging that the duration cis smaller than the target time X (step S434: No), regards L as L+1(step S436), further repeatedly exerts the operation from the step S428.And, when judging the duration c is bigger than the target time X (stepS434: Yes), the random access processing unit 400 judges that an erroroccurs (step S438).

After specifying a target sample in the step S402, the step S410, thestep S420 or the step S432, the random access processing unit 400outputs the target sample identification information indicating thetarget sample (step S440).

In this way, this variation makes it possible to reduce the calculationamount compared to the operation of the conventional random accessprocessing unit 812 shown in FIG. 8 and quickly search a target samplebecause of the following reason: First, the moof duration field 131 dthat stores the moof duration information is established in the mfhd 131of the moof 130′ in the fragmented data part and the traf duration field135 that stores the traf duration information is established in the tfhd133; second, the fragmented content duration is used when searching thetarget sample using the target time when performing a random access;third, the moof duration shorter than the fragmented content duration isused; fourth, the traf duration shorter than the moof duration is used;and lastly, a sample duration shorter than the traf duration is used.

After that, the random access processing unit 400 outputs the MP4 filedata 100′ and the target sample identification information to thedecoding unit 213, and has the decoding unit 213 and the playback unit214 exert playing back the content from the target sample indicated inthe target sample identification information.

FIG. 31 is a display diagram showing the syntax of the tfhd 133.

As shown in FIG. 31, for example, 32 bit is assigned to the trafduration field 135 of the traf 132.

The main control unit 410 of the random access processing unit 400judges the output condition of the moof duration information from themoof duration processing unit 420 and the effectiveness of the moofduration indicated in the moof duration information, and it may use theaddition result of all the traf durations instead of the moof durationby making the traf duration processing unit 430 output all pieces of thetraf duration information stored in the moof 130′ when using the moofduration, when the moof duration information is not outputted, or whenthe moof duration is judged to be uneffective. In addition, the maincontrol unit 410 of the random access processing unit 400 judges theoutput condition of the traf duration information from the traf durationprocessing unit 430 and the effectiveness of the traf duration indicatedin the traf duration information, and it may use the addition result ofall the sample durations instead of the traf duration by making thesample duration processing unit 440 output all pieces of the sampleduration information stored in the traf 132 when using the trafduration, when the traf duration information is not outputted, or whenthe traf duration is judged to be uneffective.

Also, flags indicating whether or not respective samples are seekableare contained in the above-mentioned variation, a single flag for pluralsamples corresponding to the moof 130 or a single flag for pluralsamples corresponding to the traf 132 can also be contained. In thisway, the checking process of the flags for respective samples can beomitted.

Industrial Applicability

The demultiplexing apparatus concerning the present invention issuitable for a personal computer or a cellular phone that downloads andplays back the MP4 file data, and the multiplexing apparatus concerningthe present invention is suitable for a digital video camera or acellular phone with a videotaping function that obtains multi media datasuch as video or audio, makes and videotapes the MP4 file data.

1-4. (canceled)
 5. A demultiplexing apparatus for obtaining anddemultiplexing multiplexed data where contents of a digital copyrightedwork are multiplexed starting from a header of the multiplexed data,wherein the multiplexed data is made in a way that plural pairs of acontent storage box for storing a part of multiplexed contents and arelated information storage box for storing information relating to thepart of the multiplexed contents are arranged serially, and that totalduration information for specifying a total duration needed for playbackof all the contents or a predetermined description except the totalduration is contained in the header of the multiplexed data, thedemultiplexing apparatus comprising: an obtainment unit operable toobtain the multiplexed data; a demultiplexing processing unit operableto demultiplex the multiplexed data obtained by the obtainment unit foreach pair; a total duration specification unit operable to specify thetotal duration based on the total duration information contained in theheader of the multiplexed data obtained by the obtainment unit; and adisplay unit operable to display the total duration specified by thetotal duration specification unit, the total duration specification unitjudges whether or not the multiplexed data obtained in the obtainmentunit contains the total duration information, and further judges whetheror not the total duration information indicates the predeterminedinformation in the case of judging that the multiplexed data containsthe total duration information, the total duration specification unitspecifies the total duration indicated in the total duration informationin the case of judging that the total duration information does notindicate the predetermined information, the total duration specificationunit judges that it is impossible to specify the total duration in thecase of judging that the multiplexed data does not contain the totalduration information or in the case where the total duration informationindicates a predetermined information, and the display unit notifiesthat the total duration is unknown in the case where the total durationspecification unit judges that it is impossible to specify the totalduration.
 6. A multiplexing apparatus for multiplexing contents of adigital copyrighted work and generating multiplexed data, comprising: anobtainment unit operable to obtain the contents; a multiplexing unitoperable to multiplex the contents; a duration information generationunit operable to generate total duration information which indicates atotal duration needed for playback of all the contents or that the totalduration is unknown based on the contents obtained by the obtainmentunit; and a data generation unit operable to generate the multiplexeddata in a way that plural pairs of a content storage box for storing apart of the contents multiplexed by the multiplexing unit and relatedinformation storage box for storing information relating to the part ofthe contents are arranged serially and so that the total durationinformation generated by the duration information generation unit iscontained in a header.
 7. A demultiplexing apparatus for obtaining anddemultiplexing multiplexed data where contents of a digital copyrightedwork are multiplexed starting from a header, wherein the multiplexeddata is made in a way that plural pairs of a content storage box forstoring a part of the multiplexed contents and related informationstorage box for storing information relating to the part of the contentsare arranged serially and that the total duration information forspecifying total duration needed for playback of all of the contents iscontained in the header, and the multiplexed data consists of a firstpart or a first part and a second part including one or more said pairsdepending on the contents, the total duration information indicates timeduration needed for playback of contents contained in the first part inthe case where the multiplexed data consists of the first part, and thetotal duration information consists of a first duration informationwhich indicates duration needed for playback of the contents containedin the first part and second duration information which indicatesduration needed for playback of the contents contained in the secondpart or that the duration is unknown in the case where the multiplexeddata consists of the first part and the second part, the demultiplexingapparatus comprises: an obtainment unit operable to obtain themultiplexed data; a demultiplexing processing unit operable todemultiplex the multiplexed data obtained by the obtainment unit foreach pair; a total duration specification unit operable to specify thetotal duration based on the total duration information contained in aheader of the multiplexed data obtained by the obtainment unit; and adisplay unit operable to display the total duration specified by thetotal duration specification unit, the total duration specification unitjudges whether the multiplexed data obtained in the data obtainment unitconsists of the first part, or the first part and the second part, thetotal duration specification unit specifies duration indicated by thetotal duration information as the total duration in the case where itjudges that the multiplexed data consists of the first part, the totalduration specification unit further judges whether duration needed forplayback of the contents contained in the second part is unknown or notbased on the second duration information in the case where it judgesthat the multiplexed data consists of the first part and the secondpart, the total duration specification unit specifies the total durationby adding duration indicated by the second duration information toduration indicated by the first duration information in the case whereit judges that duration is not unknown, the total duration specificationunit judges that it cannot specify the total duration in the case whereit judges that duration is unknown, and the display unit notifies thatthe total duration is unknown in the case where the total durationspecification unit judges that it cannot specify the total duration. 8.The demultiplexing apparatus according to claim 7, wherein the header ofthe multiplexed data contains part specification information indicatingwhether the multiplexed data consists of the first part and the secondpart or not, and the total duration specification unit judges whetherthe multiplexed data obtained by the obtainment unit consists of thefirst part, or the first part and the second part based on the partspecification information. 9-21. (canceled)
 22. A multiplexing apparatusfor multiplexing contents of a digital copyrighted work and generatingmultiplexed data, comprising: an obtainment unit operable to obtain thecontents; a multiplexing unit operable to multiplex the contents; aduration information generation unit operable to generate total durationinformation for specifying total duration needed for playback of thecontents based on the contents obtained by the obtainment unit; a datageneration unit operable to generate the multiplexed data in a way thatplural pairs of a content storage box for storing a part of the contentsmultiplexed by the multiplexing unit and related information storage boxfor storing information relating to the part of the contents arearranged serially and so that the total duration information generatedby the duration information generation unit is contained in a header,wherein the data generation unit makes the multiplexed data from a firstpart, or a first part and a second part including one or more said pairsaccording to the contents contained in the multiplexed data, theduration information generation unit generates the total durationinformation as the information indicating a duration needed for playbackof contents contained in the first part in the case where themultiplexed data consists of the first part, and the durationinformation generation unit generates the total duration informationfrom first duration information indicating a duration needed forplayback of contents contained in the first part, and the secondduration information which indicates a duration needed for playback ofcontents contained in the second part or that the duration is unknown inthe case where the multiplexed data consists of the first part and thesecond part.
 23. The multiplexing apparatus according to claim 22,wherein the data generation unit generates the multiplexed dataincluding part specification information indicating whether or not themultiplexed data is made from the first part and the second part in theheader of the multiplexed data.
 24. A multiplexing demultiplexingapparatus for obtaining multiplexed data where contents of a digitalcopyrighted work are multiplexed starting from a header of themultiplexed data and demultiplexing the multiplexed data, wherein themultiplexed data is made in a way that plural pairs of a content storagebox for storing a part of multiplexed contents and a related informationstorage box for storing information relating to the part of themultiplexed contents are arranged serially, and in a way that totalduration information for specifying a total duration needed for playbackof all the contents is contained in the header of the multiplexed data,the demultiplexing apparatus comprising: an obtainment unit operable toobtain the multiplexed data; a demultiplexing processing unit operableto demultiplex the multiplexed data obtained by the obtainment unit foreach pair; a total duration specification unit operable to specify thetotal duration based on the total duration information contained in theheader of the multiplexed data obtained by the obtainment unit; adisplay unit operable to display the total duration specified by thetotal duration specification unit, a playback unit operable to play backthe contents demultiplexed by the demultiplexing processing unit; and aplayback time specification unit operable to specify playback time ofthe contents being played back by the playback unit, the display unitcalculates a ratio of playback time, which is specified by the playbacktime specification unit, to the total duration which is specified by thetotal duration specification unit and displays a predetermined graphicalprogress indicator that is moved according to the ratio. 25-29.(canceled)